# Continuous pre-training optimization

Llama 3.3 Swallow is a large language model (70B) based on continuous pre-training of the Meta Llama 3.3 model, enhancing Japanese capabilities while retaining original English proficiency.

The Gemma-2-Llama-Swallow series of models are multilingual large models constructed through continuous pre-training based on Gemma-2, with a particular enhancement in Japanese ability.

A large language model specifically designed for processing ultra-long text sequences (supporting up to 1 million, 2 million, and 4 million tokens), maintaining excellent performance in standard benchmarks

SurgicBERTa is a language model developed based on the RoBERTa-base architecture, specifically optimized for surgical textbooks and academic papers.